PENSE: A Penalized Elastic Net S-Estimator

Penalized regression estimators have been widely used in recent years to improve the prediction properties of linear models, particularly when the number of explanatory variables is large. It is well-known that different penalties result in regularized estimators with varying statistical properties. Motivated by the analysis of plasma proteomic biomarkers that tend to form groups of correlated predictors, we focus here on estimators with an Elastic Net penalty, in order to keep these groups of variables together as they enter or leave the model. Given the presence of potential outliers in our data, we propose a class of penalized S-estimators which have very good robustness properties. Furthermore, these penalized S-estimators can be used as initial values to compute more efficient penalized M-estimators. In this paper we derive an algorithm to compute our proposed estimators, and also a data-driven method to select the penalty term, which is a critical part of any application with real data. Our robust penalized estimators have very good robustness properties and are also consistent under relatively weak assumptions. Our numerical experiments show that our proposals compare favourably to other robust penalized estimators. When applied to our motivating example, the robust estimators identify new potentially relevant biomarkers that are not found with non-robust alternatives. Moreover, the robust estimators identify two patients with a suspected low obstruction in the artery examined. Further measurements by a more accurate technique validated our predictions. 1